Detecting spam search results for context processed search queries

ABSTRACT

A programmable search engine system is programmable by a variety of different entities, such as client devices and vertical content sites to customize search results for users. Context files store instructions for controlling the operations of the programmable search engine. The context files are processed by various context processors, which use the instructions therein to provide various pre-processing, post-processing, and search engine control operations. Spam related and biased contexts and search results are identified using offline and query time processing stages, and the context files from vertical content providers associated with such spam and biased contexts and results are excluded from processing on direct user queries.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation of application of, and claims benefit under 35U.S.C. §120 to U.S. patent application Ser. No. 11/202,382, which wasfiled on Aug. 10, 2005, the entire contents of which are herebyincorporated by reference.

This application is related to the following patent applications, thedisclosures of which are incorporated herein by reference:

-   U.S. patent application Ser. No. 11/202,423, filed on Aug. 10, 2005,    for “Programmable Search Engine”;-   U.S. patent application Ser. No. 11/202,410, filed on Aug. 10, 2005,    for “Sharing Context Data Across Programmable Search Engines”;-   U.S. patent application Ser. No. 11/202,383, filed on Aug. 10, 2005,    for “Aggregating Context Data For Programmable Search Engines”;-   U.S. patent application Ser. No. 11/201,754, filed on Aug. 10, 2005,    for “Generating and Presenting Advertisements based on Context Data    for Programmable Search Engines”;-   U.S. patent application Ser. No. 10/921,381, filed Aug. 18, 2004,    for “Method for Detecting Link Spam in Hyperlinked Databases”; and-   U.S. patent application Ser. No. 11/004,250, filed Dec. 3, 2004, for    “Method and System to Detect E-mail Spam Using Concept    Categorization of Linked Content”.

FIELD OF INVENTION

This invention relates in general to search engines, and moreparticularly, to search engines that are programmable by clients, hosts,and other devices and systems that make use of the search engine'sservices.

BACKGROUND OF INVENTION

The development of information retrieval systems has predominantlyfocused on improving the overall quality of the search results presentedto the user. The quality of the results has typically been measured interms of precision, recall, or other quantifiable measures ofperformance. Information retrieval systems, or ‘search engines’ in thecontext of the Internet and World Wide Web, use a wide variety oftechniques to improve the quality and usefulness of the search results.These techniques address every possible aspect of search engine design,from the basic indexing algorithms and document representation, throughquery analysis and modification, to relevance ranking and resultpresentation, methodologies too numerous to fully catalog here.

Regardless of the particular implementation technique—the fundamentalarchitectural assumption for search engines has that the search engine'soperational model is fixed and non-alterable by entities external to thesystem itself. That is, the search engine operates essentially as a“black box”, which receives a search query, processes the query using acomplex, yet preprogrammed search algorithm and relevance ranking model,and provides the search results. Even where the details of the searchalgorithm are publicly disclosed, the search engine itself stilloperates only according to this algorithm, and nothing more.

An inherent problem in the design of search engines is that therelevance of search results to a particular user depends on factors thatare highly dependent on the user's intent in conducting the search—thatis why they are conducting the search—as well as the user'scircumstances—the facts pertaining to the user's information need. Thus,given the same query by two different users, a given set of searchresults can be relevant to one user and irrelevant to another, entirelybecause of the different intent and information needs. Most attempts atsolving the problem of inferring a user's intent typically depend onrelatively weak indicators, such as static user preferences, orpredefined methods of query reformulation that are nothing more thaneducated guesses about what the user is interested in based on the queryterms. Approaches such as these cannot fully capture user intent becausesuch intent is itself highly variable and dependent on numeroussituational facts that cannot be extrapolated from typical query terms.

Consider, for example a user query for “Canon Digital Rebel”, which isthe name of a currently popular digital camera. From the query alone itis impossible to determine the user's intent, for example, whether theuser is interested in purchasing such a camera, or whether the user ownsthis camera already and needs technical support, or whether the user isinterested in comparing the camera with competitive offerings, orwhether the user is interested in learning to use this camera. That is,the user's situational facts (e.g., whether or not they own the cameracurrently, their level of expertise in the subject area), and theirinformation need (e.g., the type, form, level of detail, of the requestinformation) cannot themselves be reliably determined by either analysisof query terms, or resort to previously stored preference data about theuser.

Another method of inferring intent is the tracking and analysis of prioruser queries to build a model of the user's interests. Thus, some searchengines store search queries by individual users, and then attempt todetermine the user's interests based on frequency of key words appearingin the search queries, as well as which search results the useraccesses. One problem with this approach is the assumption that queriesaccurately reflect a user's interests, either short term or long term.Another is that it assumes that there is a direct and identifiablerelationship between a given information need, say shopping for adigital camera, and the particular query terms used to find informationrelevant to that need. That assumption however is incorrect, as the samequery terms can be used by the same (or different users) having quitedifferent information needs.

Perhaps because in part of the inability of contemporary search enginesto consistently find information that satisfies the user's informationneed, and not merely the user's query terms, users frequently turn towebsites that offer highly specialized information about particulartopics. These websites are typically constructed by individuals, groups,or organizations that have expertise in the particular subject area(e.g., knowledge about digital cameras). Such sites—referred to hereinas vertical content sites—often include specifically created contentthat provides in-depth information about the topic, as well organizedcollections of links to other related sources of information. Forexample, a website devoted to digital cameras typically includes productreviews, guidance on how to purchase a digital camera, as well as linksto camera manufacturer's sites, price comparison engines, other sourcesof expert opinion and the like. In addition, the domain experts oftenhave considerable knowledge about which other resources available on theinternet are of value and which are not. Using his or her expertise, thecontent developer can at best structure the site content to address thevariety of different information needs of users.

However, while such vertical content sites provide extensive usefulinformation that the user can access to address a particular currentinformation need, the problem remains that when the user returns to ageneral search engine to further search for relevant information, noneof the expertise provided by the vertical content site is made availableto the search engine. Many vertical content sites provide a search fieldfrom which the user can access a general search engine. This field ismerely used to pass a user's search query back to the general searchengine. However, none of the expertise that is expressed in the verticalcontent site is directly available to the general search engine as partof the user's query in order to provide more meaningful search results.The expert content developer has no formal, programmatic way of passinginformation to the general search engine that expresses their expertisein their particular knowledge site.

In other words, there are no contemporary search engines that can beprogrammed by external entities—such as vertical content sites—duringthe search process itself, in way that can enhance the search processwith the expertise of the content developer of the vertical contentsite.

SUMMARY

A user's query is processed using context information that describes anycombination of pre-processing operations (conducted prior to queryexecution) and post-processing operations (conducted on the searchresults from query execution). The pre-processing operations includeoperations to revise, modify or expand the query, to select one or moredocument collections on which to conduct the search, to set varioussearch algorithm parameters for evaluating the query, or any other typeof operation that can refine, improve, or otherwise enhance the qualityof the user's search query. The context processed query is then executedby a search engine to obtain a set of search results. Thepost-processing operations applied to the search results includeoperations to filter, organize, and annotate the search results as wellas provide links to related contexts for other types of information orinformation needs. The context processing operations can be provided bya programmable search engine site, by a vertical content provider site,or by a client device. The context processing operations are controlledby context files that include commands, parameters, and instructions.The context files may be stored at the programmable search engine site,at various vertical content providers, or at client device. Contextfiles from multiple different sources can be used jointly. Contextprocessing can also be limited to either pre-processing, orpost-processing. The selection of which context files to apply to agiven user query or a set of search results can be based on the query,the user, the client device, the vertical content site from which thequery was received. The selection may be based as well on one or moresubscriptions that a user has to particular vertical content providers,or popularity or reputation of a vertical content provider.

Spam related and biased context files, and their associated verticalcontent providers are also identified. An offline processing stageidentifies spam related context files using a spam filter to evaluatevarious sites and pages selected by a vertical content provider forinclusion in its context files. The context files from vertical contentproviders identified as being spam related are excluded from usageduring subsequent processing of user search queries directly received bythe programmable search engine. A query time processing stage applies aspam filter to the search results from a context processed query. Searchresults that are identified as spam are excluded from the contextprocessed search results provided to the user. In addition, othercontext related content, including links and annotations, from this spamrelated vertical content provided are also removed from the searchresults. The query time processing stage also preferably includes a biasdetection filter that identifies biased search results. The biasedsearch results are identified by a distance measure between the searchresults from the context processed query, and native search results.Biased search results are also filtered from the results provided to theuser.

The invention also has embodiments in computer program products,systems, user interfaces, and computer implemented methods forfacilitating the described functions and behaviors.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a page from a host domain having a search field foraccessing the programmable search engine.

FIG. 2 illustrates the results of a search from the host domain.

FIG. 3 illustrates a further page accessed from the search results page.

FIG. 4 illustrates a generalized system architecture for theprogrammable search engine.

FIG. 5 illustrates a first system architecture for a programmable searchengine.

FIG. 6 illustrates a second system architecture for a programmablesearch engine.

FIG. 7 illustrates a third system architecture for a programmable searchengine.

FIG. 8 illustrates a combined system architecture for a programmablesearch engine.

FIG. 9 illustrates a simple example of a set of context files.

FIG. 10 illustrates offline spam filtering of context files.

FIG. 11 illustrates query time filtering of context files.

The figures depict various embodiments of the present invention forpurposes of illustration only. One skilled in the art will readilyrecognize from the following discussion that alternative embodiments ofthe illustrated and described structures, methods, and functions may beemployed without departing from the principles of the invention.

DETAILED DESCRIPTION Introduction to Programmable Search

Referring now to FIGS. 1-3, there is shown an example of the userexperience in using a programmable search system in accordance with anembodiment of the present invention. In FIG. 1 there is shown a page 100from a host site, digitalslr.org, which is an example of a verticalcontent site, here the field of digital cameras, as its content andorganization reflects the viewpoint and knowledge and of the entity thatprovides the site content. A vertical content site can be on any topic,and offer any type of information, and thus is not limited in thatregard. For example, vertical content sites include sites on particulartechnologies or products (e.g., digital cameras or computers), politicalwebsites, blogs, community forums, news organizations, personalwebsites, industry associations, just to a name a few. What verticalcontent sites offer is a particular perspective and understanding of theworld, one that may be of interest and value to some users. Thisperspective and understanding can be expressed, at least in part, by thecontent provider's organization and selection of content, as well ascommentary, analysis or links to other content (e.g., commentary onother sites on the Internet). Indeed, one valuable aspect of verticalcontent sites is the particular collection of links to other sites thatthe content developer has judged to be useful in some regard, either forits depth, expertise, viewpoint, or the like. That is, users in generalfind value in the judgments of vertical content providers as to theusefulness of other sources of information on the Internet.

The host site includes a web server for serving pages, like page 100, toclient devices. The pages are stored in some repository, such as adatabase, file directories, or the like. Thus, for example, the page 100includes commentary on the latest camera offerings from variouscompanies, as well as a link 102 to another site with relevantinformation about digital cameras. Of interest in this example is thesearch field 104, which allows the user to search the Internet using ageneral search engine system (not shown), such as the Google® searchengine provided by Google, Inc. of Mountain View, Calif. (of course inother embodiments, other search engines may be used, even if they arenot nearly as powerful and sophisticated as Google). The user enters asearch query in the search field 104. Here, the query is “Nikon d100”.

Selecting the search button 106 results in the web server transmittingthe search query to the search engine system, using existing webprotocols. In this example embodiment, in addition to the search query,the host site web server transmits a context file to the search enginesystem (alternatively, the web server can transmit a link to the contextfile, or simply a context file identifier). The context file includesdata that the search engine system uses to control the operation of thesearch engine itself in processing the search query and in presentingthe search results, in effect, programming the search engine'soperation. Thus, the context file, as will be further detailed below,can be understood as a set of instructions to the search engine systemfor processing a particular search query. The instructions can controlthree aspects of the search process: 1) pre-query processing operations;2) search engine control information; 3) post-query processingoperations. Among other aspects, a context file may include descriptionsof (or links to) related context files, which likewise provide furtherprogrammatic control of the search engine system.

FIG. 2 illustrates an example a search results page 200 that is providedback to the user's client device, following processing of the contextfile and the search query. This page 200 includes a set of searchresults 202 that satisfy the search query, as well as additionalinformation. First, there is displayed a name of the current context 208that has been provided to the search engine system; the name is simply adescription the vertical content site developer's has given to expressthe type of information need or contextual circumstances that pertainsto the current search query. Here, for example the current context 208is for a “Camera Model”, since the search query matched a specificcamera model name as determined by processing of the context file. Thiscontext operates as the entry point for a user seeking information abouta particular camera model.

Second, a number of links 204 are provided as navigational aids tofurther pages that address different possible information needs of theuser. Each of these links 204 is associated with a related context file,which will provide further instructions to the search engine system totailor further stages in the search process for a specific informationneed, and thereby construct the desired pages. For example, the firstlink “If you are trying to decide which camera to buy” addresses aspecific type of user information need—information about how to purchasea camera, comparisons between camera, pricing information, and thelike—that derives from a specific type of user intent, the intent topurchase a camera. The second link “Where to buy this camera from . . .” addresses a different and more specific information need, the locationof vendors for that particular camera. The last link “If you already ownone . . . ” address yet entirely different type of information need,that is for information that a current own would want, such as technicalsupport and service information, information that is not relevant to thetwo previous information needs.

Third, the page 200 includes links 206 to other related contexts aswell, such as “More Manufacturer Pages”, “More Guides”, “More Reviews”,and so forth. These links each invoke a particular context in which thevertical knowledge provider has characterized particular sites andpages, and then defined a filter for the search engine to select pageswith the matching characteristics when processing the reformulatedsearch query.

For example, the vertical knowledge provider has here previouslyidentified a number of different sites or pages on the internet as beingvariously manufacturer sites, product review, buying guides, and soforth (e.g., according to the type of site). The vertical knowledgeprovider can label (or tag) a site with any number of category labels.The labels can describe any characteristic that the vertical contentprovider deems of interest, including topical (e.g., cameras, medicine,sports), type (e.g., manufacturer, academic, blog, government), level ofdiscourse (e.g., lay, expert, professional, pre-teen), quality ofcontent (poor, good, excellent), numerical rating, and so forth. Theontology (i.e., set of labels) used by the vertical knowledge providercan be either proprietary (e.g., internally developed) or public, or acombination thereof.

For example, the vertical site provider has previously identified anumber of sites as containing product reviews, and has stored thisinformation in a context file. The link 206 to “More reviews”automatically would control the system engine system to use this contextfile to filter the search results during post-processing to those pagesthat are from sites characterized as product reviews, and satisfying thereformulated query.

Fourth, the page 200 includes various annotations 210 in conjunctionwith various ones of the search results. These annotations 210 providethe user with the viewpoint or opinion of the vertical knowledgeprovider about the particular search result, as to any aspect of thatsearch result that the provider considers significant, such as what theidentified search result is about, how useful it is, or the like.

The placement, naming, and sequencing of the various links 204, 206 arethemselves defined in the context files. This gives the verticalknowledge content provider almost complete control over the organizationand presentation of the search results, which in and of itselfrepresents that provider's particular perspective and determination ofwhat are the user's likely information needs, and how the search resultsshould be organized to satisfy those needs, and which related contextsshould appear in response to each level of search by the user.

FIG. 3 illustrates an example page 300 that is provided to the user as aresult of clicking on the first link 204, “If you are trying to decidewhich camera to buy.” The context file associated with this link 204 isprocessed, and a second search is performed on the search query. Thispage 300 shows the context name 308 “Choosing a camera”, which againreflects the selected information need of the user. The search results302 in this context are more specifically tailored to assisting the userin evaluating digital cameras and selecting a satisfactory one. Notice,for example, the first search result is to a buying guide for digitalcameras, and that there are no search results shows shown here totechnical support pages.

Above the search results 302 are links 304 to further related contextsbased on information needs, such as “Review, sample photographs”, “Othersimilar cameras to consider”, and “Relevant product news”. Again, theselinks have associated context files that will control the search enginesystem to provide search results that are relevant to the describedinformation needs for these contexts. Next to the search results areadditional links 306, which are also to related contexts, and forexample to further professional and user reviews of digital cameras,sample photographs, and other information particularly relevant toevaluating a camera for purchase.

The user can thus continue to access additional related context throughthe various links 304, 306, each time obtaining search results that havebeen processed according to the context files associated with theselected links. In this way, the user can essentially search theInternet using the powerful capabilities of a general search engine, yetobtaining the benefit of the knowledge, expertise, and perspective ofthe provider of the vertical content site. Vertical content siteproviders benefit from this approach as it allows them to further sharetheir knowledge and perspective with users. Vertical content providersare no longer limited to the information that they can either createthemselves, provide links to, or comment upon.

With the capabilities of the present invention, vertical contentproviders can define any variety of context files to meet any type ofinformation need that users may have. The providers of the generalsearch engine system are no longer burdened with the task of themselvesorganizing and categorizing content (as is conventionally done invarious directories and portals), but instead can rely upon the muchdeeper and vaster pool of vertical content providers—hundreds ofmillions or more—as compared with the limited pool of editors that mayorganize content directories or categorize other websites for a generalsearch engine. Indeed, no individual search engine provider couldpossibly employ the number of individuals with sufficient breadth anddepth of experience, knowledge, or perspectives to itself provide thescope and variety of contexts that exists across the entire Internetcommunity. Instead, the present invention provides any vertical contentsite provider with the capability to programmatically control thegeneral search engine system on behalf of a user conducting a search.

The foregoing example is but one possible use of the present invention,and many more applications and usages will become apparent in thefollowing discussion.

In another manifestation, the context file need not change the order inwhich the initial results are presented, but only annotate the resultswith the labels (e.g., tags) that apply to them. Clicking on the labelissues a new search, which is restricted to the results having metadatamatching the label. In yet another manifestation, these annotations neednot be labels but links to relevant pages on other sites.

In yet another manifestation, the query need not originate at thevertical content site, but at the search engine's site, but use theknowledge provided by the vertical content site. In this embodiment, theuser indicates to the search engine, either while using the verticalcontent site or through a sign up process similar to that used tosubscribe to RSS feeds that the user would like to apply the verticalcontent site's contexts while conducting searches of a particular type.

System Overview

General System Architecture for a Programmable Search Engine

FIGS. 4 through 8 illustrate a number of different system architecturesin which the present invention can be employed. These architecturesgenerally vary in terms of which entities provide the context files andwhich entities processes the context files to control the search processand search result presentation. In general, the context files can beprovided by any system entity (e.g., any of a client device, a hostvertical site, or the search engine system), and can likewise byprocessed by any system entity, or any combination there.

Referring first then to FIG. 4, there is shown a generic systemarchitecture. In this system architecture, there is a client device 402,a vertical content server 406, context server 430, a context processor420, and a programmable search engine (PSE) 450.

The client 402 can be any type of client, including any type of computer(e.g., desktop computer, workstation, notebook, mainframe, terminal,etc.), handheld device (personal digital assistant, cellular phone,etc.), or the like. The client device 402 need only have the capabilityto communicate over a network (e.g. Internet, telephony, LAN, WAN, orcombination thereof) with the PSE 450. Typically, a client device 402will support a browser application, and the appropriate networkingapplications and components, all of which are known to those of skill inthe art. The client device 402 may include as well a search engineinterface that also it to directly query the PSE 450.

The user of the client 402 constructs and transmits a search query tothe PSE 450, via the content server 406, which includes a search engineinterface (SEI) 409. This interface can be in part, as illustrated inFIG. 1, via a search query field on a host site that includes thecontent server 406, along with an underlying link to initiate processingof the input text and forwarding the results thereof to the PSE 450. Thecontent server 406 selects an appropriate context file, as identified bya context ID. The selection of the context file can be based on thequery itself, the client device 402, the user identification, defaultselection parameters, user site behavior (e.g., page accesses, dwelltimes, clicks) or other information programmatically available to thecontent server 406. The context ID may be a URL, a unique context name,a numerical ID, or some other form of reference to the context file. Thecontent server 406 transmits the query along with the context ID to thecontext processor 420. Alternatively, content server 406 can provide theidentified context file directly to the context processor. Depending onthe embodiment, the content server 406 may also be responsible forserving content pages to the client device 402.

The context processor 420 uses the context ID to obtain the identifiedcontext file from the context server 430. The context processor 420 mayalso pass an identifier of the client device 402 (e.g., IP address,browser type, operating system, device type), the user (e.g., user ID),or host domain from which the search query is received, or the searchquery itself, to obtain further context files from the context server430.

As discussed above, a context file (or collection of context files) caninclude three types of programmatic information that can be used by thecontext processor 420 and/or PSE 450 to control the search process.These are: 1) pre-query processing operations; 2) search engine controldata; 3) post-query processing operations. This programmatic informationwill be discussed as part of the operational flow.

The context processor 420 processes the context files to perform variouspre-processing operations, to programmatically generate a reformulatedquery. These pre-processing operations may be performed independently orin any combination to obtain a reformulated query. These include thefollowing:

a) Query revision: the modification, addition, or deletion of or one ormore terms of the original query. Such modifications include correctingspelling errors, including replacing query terms, adding query terms (asconjuncts, or as disjuncts) or deletion of query terms (e.g. stop wordremoval). The added or replaced terms may broaden or narrow the scope ofa query.

b) Creation of additional queries: For example, given an original searchquery of “digital SLR”, an additional query may be “digital camera”.These types of query reformulations are expressed in the context file asa series of query rewrite rules. The query rewrite rules generallydefine an output query (or query term) based on matching one or moreterms of the original query (e.g., replace “digicam” with “digitalcamera”). Other rules may be applied automatically as defaults, withoutbeing conditioned on the terms of the query.

The second type of control information processed by the contextprocessor 420 are search engine control data. These include:

a) selection of one or more search engines for processing thereformulated search query. The PSE 450 may include any number ofdifferent search engines, each of which is optimized for certain typesof searches. For example, different search engines are typically usedfor text searches, image searches, and audio searches. A search enginetypically will generate an information retrieval score for variousdocuments in terms of their relevance to the search query. A contextfile can specify which search engine(s) is to be used (e.g., byidentification of a particular URL for the search engine). The contextprocessor 420 extracts this identified search engine, and constructs theappropriate query string using the reformulated query.

b) selection of one or more document collections for searching. A searchengine system will typically have access to multiple different documentcollections, which can be searched jointly, or individually. Theprovider of the context file may instruct the PSE 450 to use one or morespecific document collections for a particular search. For example, avertical content site for healthcare professional, may receive a searchfor “migraine”, and instruct the search engine system to search thePubMed database provided by the National Library of Medicine, ratherthan a more general search of the Internet. This constraint bettertailors the results to the medical literature most likely to be relevantto the information need of a healthcare professional, rather than thetypical results to such a query on the Internet. The context file canspecify which document collections are to be used (e.g., byspecification of a database, index, or other context repository). Thecontext processor 420 extracts this information from the context file aswell, and passes it the selected search engine as a parameter.

c) specification of search engine parameters for use during queryprocessing. Most search engine algorithms operate under a large numberof parameterized controls when generating information retrieval scores,such as threshold values for scoring query term matches, iterationcycles, waiting of links, terms and other query or document attributes.Normally, these parameters are not accessible to entities outside of thesearch engine system, but rather are fixed by the search engineprovider. However, in some embodiments of the present invention, thesearch engine system may be configured to receive and use any of thesetypes of parameters, thereby giving further incremental programmaticcontrol of the search engine to the vertical knowledge developments.Again, the context processor 420 extracts these parameters from thecontext file and passes them to the PSE 450 as parameters.

The context processed query, which includes the reformulated query andthe search engine control data (if any) that are specified in thecontext file, is thus provided to the PSE 450. If multiple queries areconstructed during pre-processing, the context processor sends each ofthe multiple queries and their associated search engine control data(which may be individually varied for each additional query).

The PSE 450 processes the reformulated query using the search enginecontrol data (if any) to obtain a set of context processed searchresults, and provides these search results back to the context processor420. If multiple queries are processed, then the PSE 450 can merge theresults from these searches.

The context processor 420 then provides various post-processingoperations, which again may be performed independently or conjointly.The results of this post-processing made part of the context processedsearch results. The post-processing operations include:

a) filtering the context processed search results using filtersspecified in the identified context. The context file may specify one ormore filters that the context processor 420 can apply to further limitthe documents that are included in the search results. These filters areexpressed in terms of rules that match metadata with particular metadataassociated each search result. The metadata can include both nativemetadata to the document, such the document type, date, author, site,size, or labeled metadata associated with the document, that is thelabeled characteristics provided by the vertical content provider (orothers).

For example, the filters may be defined to exclude documents of certaintypes (either physical types, e.g., image files, or logical types, e.g.,“reviews”), from particular sites or internet domains (e.g., documentsfrom the .biz or .gov domain), websites, or of a certain vintage (e.g.,documents published before Mar. 3, 2005). Referring back then to theexample of FIG. 3, the link 306 for “More Professional reviews” wouldinvoke a filters defined to select only documents labeled as“professional”, “product reviews”. Again, these labels can be providedby the vertical knowledge content provider from which the original querywas sourced, or from some other source. These options will be more fullydiscussed below.

b) ranking of the context processed search results using rankingparameters specified in the context file. The PSE 450 includes a rankingfunction that ranks the search results based on the respectiveinformation retrieval scores. The context file can include rankingparameters, such as weighting factors to increase or decreases the IRscores for particular types of documents, for documents from selectedsources. The ranking function may also operate on identifiable native orlabeled metadata. For example, the rankings can be adjusted based onlength of document, publication date, or document format just to name afew. Alternatively, the ranking may be adjusted based on labeledmetadata, such ranking by expressed “rank” value, or by as increasingthe native ranking of documents labeled as “expert” by a weight factor,or increasing the ranking of documents having a some specified qualitymeasure of “10”. The context processor 420 can use these rankingparameters to rank the documents in the search results.

c) clustering of the search results using clustering parameters. Thecontext processor 420 may also cluster (group) the search resultsaccording to parameters provided in the context file. The parameters canspecific clustering based on native or labeled metadata. Thus, alldocuments labeled as “professional reviews” can be clustered together;or all documents where are image files can be clustered, or documentsfrom a given domain (e.g., all documents from xxxx.com).

d) providing navigational links in the context processed search resultsto additional contexts. As illustrated in FIGS. 2 and 3, the contextprocessor may also provide links that can be accessed to invokeadditional searches for further refinements of the information needs ofthe user. Each such related context link invokes another cycle ofpre-processing and/or post-processing by the context processor 420 andif so instructed, another cycle of query processing by the PSE 450.

e) annotating the context processed search results using annotationsspecified in the identified context. As illustrated in FIGS. 2 and 3,the context file may also provide specific annotations 210 that can beincluded with any of the search results.

The context processor 420 then provides the context processed searchresults to the client device 402. As noted, the user can access any ofthe related context links, or perform entirely new queries, again makinguse of any context files that are selected based on such queries.

The client device 402 may also query the PSE 450 directly, eitherthrough its search engine interface 403, or simply by going to thewebsite of the PSE 450 entering the query directly there. In thisscenario, context processing is still handled by the context processor420 in manner described above.

Programmable Search Engine System Based Context Processing

Referring now to FIG. 5, there is a shown a system architecture in whichthe context processing operations are provided by the PSE system itself.In this embodiment again there is a client device 502 including abrowser 503, along with a host vertical content site 504, and a PSEsystem 500. The vertical content site 504 includes a vertical contentserver 506 (e.g., a web and/or application server) and vertical contentfiles 505 (e.g., a database or directory of web pages). Also present arevertical context files 507. The vertical content site 504 also includesa search interface 509 to the PSE system 500, such as a search field andsearch button as illustrated in FIG. 1. The user accesses the verticalcontent site 504 using the browser 503, and from that site can enter asearch query to be processed by the PSE system 500. The vertical contentserver 506 processes the search query to determine a context ID for anappropriate context file, and transmits the search query and context IDto the PSE system 500. For example, the context ID can be transmits as aparameter in a URL to the PSE system 500. The vertical content site 504also includes a number of conventional components (e.g. firewalls,router, load balancers, etc.) not shown here in order to not obscure therelevant details of the embodiment.

The PSE system 500 includes a number of components. A front end server552 provides the basic interface for receiving search queries. The frontend server 552 extracts the context ID and query, and passes that to acontext processor 520. The front end server 552 may also provide anidentifier of the client device or the user to the context processor520. The context processor 520 provides the context ID and query, to thecontext server 530. The context server 530 uses the context ID toretrieve a context file from a repository of cached context files 540.The context files are received from any vertical content site 504,including the illustrated site 504, via a registration interface 560.This allows any provider of a vertical content site 504 to define thecontext files that are to be used for handling queries from their siteand upload such context files for storage by the PSE system 500.Alternatively, the context files are extracted from the vertical contentsites 504 by a context file web crawler 580. The registration andcrawling methods may be used together. One implementation would be forthe vertical content site 504 to first register its context files 507,which includes putting the site address on a crawl list. Subsequently,the crawler 580 crawls the site 504 to obtain any updates to the contextfiles 507. Caching of the context files ensures very high speedprocessing of the context files at query time, since context processor520 does not need to retrieve the context files from the remotelyvertical content site 504, and thereby does not incur network latency(or problems with the vertical content site being unavailable).

The context server 530 may also obtain context files from a repositoryof global context files 542. These context files can be derived fromdata mining on the cached context files 540, provided by the provider ofthe PSE system 500, or any combination thereof.

The context server 530 provides the retrieved context file(s) to thecontext processor 520. The context processor 520 performs theappropriate pre-processing operations (if any) as defined in the contextfile to generate the reformulated query, and establish the search enginecontrol data as set forth above, as part of the context processed query.The search engine 550 receives the context processed query, includingreformulated query and search engine control data, and executes a searchon same to provide a set of context processed search query results.These results are passed back to the context processor 520, whichperforms the post-processing operations on the search results as definedin the context file, to further modify the context processed searchresults. These processed results are then transmitted back to the clientdevice 502.

This architecture provides various benefits. First, as pointed itprovides for high speed access to the context files and eliminatesreliance on the availability of the remote vertical content sites toserve their context files on demand.

Second, collection and aggregation of the context files 520 allows forvarious systemic to be achieved from analysis of the context files. Itmust be appreciated that over time, the number of vertical contentproviders employing context files will easily reach millions if nothundreds of millions, given the breadth and depth of the Internet. Thereare currently over 200,000,000 Internet sites, and that number isincreasing at a rate of more than 10% per year. Even if only 1% ofvertical content providers used context files, that would exceed2,000,000 such collections of context files, providing a very richrepository of information.

Specifically, the following types of information may be aggregated fromthe collected context files. The rules used to define the querypre-processing operations can be accumulated and used to identify themost frequently used rules for various query terms. To a large extentthis type of information is more reliable, having been essentialityvoted on by a large population of interested providers, as opposed torules designed by a very small team of editors.

Similarly, analysis of the search engine control yields identificationof most frequently used search engines, indices, and parameters forparticular queries or types of queries. Analysis of the querypost-processing operations also identifies the most frequently usedannotations, related contexts, ranking and filtering operations.

As mentioned above the context files includes label metadata used by thevertical knowledge content providers to describe the characteristics ofany site or page on the Internet. In one embodiment, these labels areselected from a publicly provided ontology, so that vertical knowledgecontent providers use the same set of labels to characterize the contentof the Internet. The ontology of labels can describe categories andinstances of any type. The ontology includes, for example, topics,information types, information sources, user types, and rating scales,just to name a few possible aspects of the ontology. Accordingly, fromthe cached context files 540 a categorization of Internet content can bederived and validated. By way of simple example, all Internet siteslabeled as type “buying guide” and category “digital camera” can beextracted from the cached context files 540. A directory of thesedigital camera buying guides can then be constructed, for example byselecting those sites having that have a minimum number of appearancesin the context files. This approach again leverages the collectivejudgment of the vertical content providers—that is, the wisdom ofcrowds—as to the nature, type, and quality of content on the Internet.

From the foregoing, the PSE system 500 can extract and establish acollection of globally optimized context files, where the querypre-processing rules, search engine control data, and querypost-processing rules are derived from statistically analysis of cachedcontext files for the frequency, distribution, variability and othermeasures of the usage of context information.

One scenario for this architecture is to support direct search querieswith post-query context processing. In this embodiment, a user query isreceived directly from the client device 502, without first being passedthrough a vertical content provider site 504. The user's search querycan be received directly at the website of the PSE system 500 (e.g., viasearch query page), or a search interface in browser toolbar,application, or system extension (e.g., a search interface on the user'sdesktop). In any event, the user's search query is handled withoutcontext based preprocessing, (that is query modification based on avertical content provider's context files), though internal adjustmentof the search query may be performed as part of native searchoperations. However, the search results are then post-processed with oneor more context files, to provide the various types of navigationallinks, related context links, and/or annotations on search results asdescribed and illustrated in FIGS. 2 and 3.

Another beneficial aspect of this architecture is that analysis of thecontext files also allows for integration of advertisement purchasesbased on contexts. That is, advertisers can bid for placement of theiradvertisements in specific contexts, rather than by specific queryterms. For example, an advertiser may bid for placement of anadvertisement for its digital camera when the context file for a queryindicates that the user is shopping for a particular camera model, butnot when the user is seeking technical support. This allows advertisersto more precisely focus their advertising efforts based on the user'sinformation needs—which have been expressly described by the contextfiles, rather than merely inferred from the query terms.

Vertical Content Provider Based Context Processing

Referring now to FIG. 6, there is shown an embodiment of a systemarchitecture in which the context processing is provided by the verticalcontent site itself. In this embodiment again there is a client device602 including a browser 603, along with a host vertical content site604, and a general search engine system 600. The vertical host verticalcontent site 604 includes a vertical content server 606 and verticalcontent files 605 (e.g., a database or directory of web pages). Thevertical content site 606 also includes a search interface 609 to thesearch engine system 600, such as a search field and search button asillustrated in FIG. 1. The user accesses the vertical content site 604and from that site can enter a search query to be processed by thesearch engine system 600.

In this embodiment, the vertical content site 604 also includes variouscomponents for context processing, including a vertical contextprocessor 620 and local vertical context files 607. As before, verticalcontent server 606 receives a search query from the client device 602,e.g., via the browser 603, and processes the search query to determine acontext ID for an appropriate context file. This information is nowprovided to the vertical context processor 620. The context processor620 passes the context ID (and optionally the client device ID, user ID,and query) to the context server 630. The context server 630 uses thecontext ID to retrieve a context file from the vertical context files607.

The context server 630 provides the retrieved context file(s) to thecontext processor 620. The context processor 620 performs theappropriate pre-processing operations as defined in the context file togenerate the context process search query (including the search enginecontrol data as set forth above). The vertical context processor 620then invokes the search engine 650 to process the context processedquery.

The search engine 650 receives the reformulated query and search enginecontrol data, and executes the search accordingly, generating thecontext processed search results. These results are passed back to thecontext processor 620, which performs the post-processing operations onthe search results as defined in the context file, to further modify thecontext processed search results. These processed results are thentransmitted back to the client device 602.

The context processor 620 may also provide some or all of the searchengine control data to the search engine, depending whether the searchengine 650 exposes an application programming interface. In someembodiment, where the search engine 650 is closed, then the contextprocessor 620 simply passes the queries to the search engine 650 andoperates on the results. In this embodiment, the context processor 620itself would use at least some of the search engine control data, forexample, selection of which search engine to use. This gives thevertical content site provider control as to which search engines 650 touse with which types of user queries.

Client Based Context Processing

Referring now to FIG. 7, there is shown an embodiment of a systemarchitecture in which the context processing is provided by the clientdevice site. In this embodiment again there is a client device 702including a browser 703, along with a host vertical content site 704,and a general search engine system 700.

As before, the vertical host vertical content site 704 includes avertical content server 706 and vertical content files 705 (e.g., adatabase or directory of web pages). The vertical content site 706 alsoincludes a search engine interface 709 to the search engine system 700,such as a search field and search button as illustrated in FIG. 1. Theuser accesses the vertical content site 704 using the browser 703 andfrom that site can enter a search query to be processed by the searchengine system 700.

In this embodiment, the client device 702 includes the variouscomponents for context processing. First, the client device 702 includesa browser 703, for accessing the vertical content site 704 as well asany other available site on the network. The client 702 includes avertical context processor 720, which can operate a plug-in to thebrowser 703, or Java applet. Here, the once the user makes the query viathe vertical content server 706, that query is also provided to thevertical context processor 720. The context processor 720 againprocesses the search query to determine a context ID for an appropriatecontext file. Since the operation is local to the browser, the contextprocessor 720 can use the context ID to retrieve a context file from theuser context files 707.

The context processor 720 then performs the appropriate pre-processingoperations as defined in the context file to generate the contextprocessed query. The vertical context processor 720 then invokes thesearch engine 750 to process the context processes query. The searchengine 750 receives the context processed query, and retrieves searchresults, forming the context processed results. These results are passedback to the context processor 720, which performs the post-processingoperations on the search results as defined in the context file, tofurther modify the context processed search results. These processedresults are then passed back to the browser 703.

An advantage of this architecture is that it allows the user toestablish and user their own context files. Just as individual verticalcontent providers have their individual expertise and viewpoint, so todo individual users. Thus, a user may define context files to categorizeand label particular websites, for example, identifying the site thatshe considers most authoritative or useful for particular topics. Theuser can also define query pre-processing operations, or more likelyimport such operations from others (e.g., experts in various topicaldomains) who publish context files for this purpose. Similarly, the usercan define post-processing operations that allow for customization inthe presentation of results, including arrangement of results intoclusters or grouping that the user feels most comfortable with. Forexample, a user can define a personal context file in which searchresults are always clustered into academic (.edu), government (.gov),retail shopping (sites having metadata or text indicative of onlinepurchasing), and image files.

Unified Architecture for Mutual Context Processing

The various architectures illustrated in FIGS. 4-7 can all operateconcurrently with different types of the individual systems operatingtogether. FIG. 8 illustrates this system architecture for mutual andconcurrent context processing. All of the system elements communicatevia a network 892, such as the Internet.

First, the PSE system 800 includes a complete set of components asdescribed with respect to FIG. 4. The operative features of thesecomponents have been previously described and so are not repeated here.

Next, three types of client devices 802 are in operation. Client device802 a simply has a browser 803 by which it accesses various sites on theInternet. Client device 802 b includes a browser 803, as well as usercontext files 807, which can be passed to any available contextprocessor 820 for processing in conjunction with a search query providedby the user.

Client device 802 c includes a browser 803 and user context files 807,as a well its own context processor 820. This enables the client 802 cto perform local context processing on the user's search query prior tosending the query to the search engine, as well as performingpost-processing operations after receiving the search results. Thisclient's browser 803 also includes a search engine interface 809,enabling direct querying of the PSE system 800. It is contemplated (butnot illustrated) that the other clients 802 a and 802 b may also includesearch engine interfaces 809, for example, in the toolbar of theirrespective browsers 803.

The three types of different vertical content sites 804 are also shown.Vertical content site 804 a includes a content server 806, along with asearch engine interface 809 to the PSE system 800, as previouslydescribed. The server forwards a user's query (from any type of theclient devices 802) to the PSE system 800, providing as well the contextID associated with the user's current context (along with any contextrelated information received from the client device). The site does notneed to store its own context files, as these can be stored at the PSEsystem 800 in the cached context file database 840.

For this type of vertical content site 804 a, the PSE system 800provides all of the context processing operations. Here, the site 804 adoes not provide any specific context ID information. As a result, thePSE system 800 can provide its own context identification mechanisms,for example based on the site 804 a, the client 802, the query terms, orthe like. Using the context information, the context server 830retrieves the appropriate global context files 842, and the contextprocessor 820 uses these files for the context processing operations,including pre-processing of the search query, control of the searchengine operation and parameters, and post-query processing. Theprogrammable search engine site 800 passes the context processed searchresults back to the requesting client, either directly, or within thescope of the vertical content site 804 b, e.g., using framingtechniques.

As with vertical content site 804 a, vertical content site 804 cincludes its own content server 806 search engine interface 809,vertical content files 805, as well as local vertical context files 807.This site 804 b receives a search query from a client device 802, andforwards the query along with the context ID for the query context tothe PSE system 800. The site's vertical context files 807 are cached inthe PSE system's cached context files 840. The PSE system 800 receivesthe context ID, and uses its context server 830 to retrieve theassociated context files for site 804 b from the cached context files840. The context server 830 may also retrieve any applicable globalcontext file 842. The PSE context processor 830 then processes theretrieved context files, generates the context processed search queryand processes the queries via the search engine 850. The contextprocessed search results are the further post-processed by the PSEcontext processor 820, again in accordance with either the site'scontext files or the global context files 842 (including whereappropriate a combination thereof).

The last type of vertical content site 802 c includes its own contentserver 806 search engine interface 809, vertical content files 805,local vertical context files 807, as well as a local, vertical contextprocessor 820. The local context processor 820 receives the user'ssearch query, along with the context ID for the user's context, andusing the referenced context files performs the appropriatepre-processing operations on the query prior to transmitting it to thePSE system 800, along with the search engine control data specified bythe context files.

Here, the PSE system 100 can provide various levels of services to thevertical content site 804 c. Minimally, the programmable search enginesystem 800 can process the received context processed queries, andexecute these queries accordingly via the search engine 850, providingthe context processed search results back to the local context processor820 for further modification. The local context processor 820 for thevertical content site 804 c provides further post-processing operationsspecified by the identified context, and then forward the final set ofcontext processed search results to the client device 802.

Alternatively, the PSE system 800 can perform some specific contextprocessing operations as instructed by the local context server 820,whether pre-processing, or post processing, or control of the searchengine operations. For example, the local context processor 820 mayperform the pre-processing operations to reform the queries, but thenuse the search engine control data to specify which document collectionsand search algorithms the search engine 850 should use. In addition, thePSE system 800 may also add its own layer of context processing based onits global context files 842, including generation of additionalreformulated queries, control of the search engine 850, andpost-processing of search results prior to returning them to thevertical content site's local context processor 820. The verticalcontent site 804 c can forward the context processed search results tothe client device 802 directly, or can invoke another layer ofpost-processing operations by the local context processor 820, perhapsto further fine tune the organization, commenting, or navigationfeatures thereof.

The PSE system 800 can provide context processing directly to userqueries input at the PSE site from any of the client devices 802. Theuser's search query can be received directly at the website of the PSEsystem 800 (e.g., via search query page), or a search interface inbrowser toolbar, application, or system extension (e.g., a searchinterface on the user's desktop). Since the user's query is not comingfrom a vertical content provider, the PSE system 800's contextprocessing can use the global context files 842, including those forannotating search results with links to potentially useful context forthe user.

The degree of context processing for direct queries can be varied, toinclude either pre-processing or post-processing individually, ortogether. One embodiment of direct query handling is providing acontext-based post-processing on the search results, without contextbased preprocessing (e.g., query modification). Here, the user's searchis received and executed without pre-processing based on the contextfiles of a specific vertical content provider (though some internaladjustment of the query and selection of search indices may be employedto provide the most relevant search results). As described with respectto FIG. 5, the search results are then post-processed with one or morecontext files, to provide the various types of navigational links,related context links, and/or annotations on search results as describedand illustrated in FIGS. 2 and 3.

The post-processing operations in this scenario can use either globalcontext files 842, or can be based on the context files of any number orselection of the vertical content providers. In one embodiment, a usercan identify which the vertical content provider whose context files areto be used for context processing. Identification can be done via asubscription model, in which the user subscribes to have such contextprocessing done for her or her queries, for example via a subscriptioninterface (e.g., page) at the website of the vertical content provider,which then forwards an identifier of the user or the user's clientdevice to the PSE 800. A user may subscriber to a particular verticalcontent provider in order to have that provider's expertise, perspectiveor viewpoint applied to the user's search queries and results, withoutthe user having to always enter a query from that vertical contentprovider's site.

For this embodiment, the PSE system 800 includes a user account database890, which stores for each user various types of personal preferencesfor searches, including the subscriptions to particular vertical contentproviders. The PSE 800 also provides a registration interface (allowingthe user to register with the PSE system 800 for storing searchpreferences, subscription information, and other user settings), and alogin interface for the user to login and have the user's settingsapplied to the user's queries. Direct queries received from the userand/or the user's client device 802 are identified by the PSE 800 andthen the appropriate context files to which the user subscribed are usedfor context processing. In another embodiment, similar to the foregoing,subscription-based context processing is provided for direct userqueries for both pre-processing and post-processing operations.

The selection of which vertical content providers' context files are tobe used (whether for pre-processing, post-processing or both) can bebased on other factors beyond a user's subscriptions, as some users maynot have subscribed to any particular vertical content provider. In oneembodiment, the selection is based on a popularity measure for eachvertical content provider whose context files are included in the cachedrepository. The popularity measure can be based on web accessstatistics, like number of unique visitors to a vertical contentprovider's site each month (or other time period), number of hits tosuch site, number of current subscribers to the vertical contentprovider. These and other statistical measures can be combined into apopularity measure. Alternatively, or additional, the selection can bebased on a reputation measure (or rank), where the reputation of eachvertical content provider is judged and rated by users.

In summary, the foregoing provides a general overview of the operationsand various system architectures useful with the present invention. Ascan be seen, the present invention can be practiced in a number ofdifferent and complementary embodiments. The capability of the presentinvention enable any system entity to provide context files, contextprocessing (or both) results in both tremendous flexibility and power.The flexibility (e.g., any system entity can provide various levels ofoperative support, and cooperate with any other system entity) allowsfor rapid, widespread and easy implementation of the present invention.The context files and context processing capability can be readilyimplemented in any vertical content site and in any client. The power ofthe system derives in part from such widespread distribution andimplementation: the more context files and context processing isadopted, the more contextual information can be accumulated andleveraged, for example in the global context files. This enables the PSEsystem to continually refine and adapt its capabilities to theinformation needs of the wide variety of users. Further, the widespreaduse of context files by vertical content developers continually expandsthe range of information needs and perspectives that can be satisfied,as well as the depth and quality of that information that is used tosatisfy such needs.

Contexts

Overview of Context File Implementation

Referring now to FIG. 9 there is shown a simple example of a set ofcontext files as might be developed by a vertical content provider for adigital camera related website. This simplified example is used only toillustrate some of the basic aspects of context files, and not asdefinitive statement of their characteristics.

In this example, the vertical content provider has provided a variety ofcontext files that suit different types of information needs, anddifferent types of available resources. Context files 902 areillustrative of contexts defined for various types of users of digitalcameras, such as a professional user searching for a digital camera, aconsumer searching for a digital camera, and an owner who already hassuch a camera. Each of these types of users has different informationneeds and typically different approaches to evaluating the informationshe obtains. For example, a professional user is typically mostconcerned with technical performance issues such as picture quality,durability, and compatibility with an existing set of professionalequipment, whereas a consumer user is typically concerned with ease ofuse, convenience and price. Both of these types of users are seekinginformation during their purchase process that is quite different froman existing owner. An owner is not typically interested in obtainingfurther opinions or evaluations of a product, but rather informationpertaining to its use, technical support, service, or warranty issues.

Each of these three user type context files 902 contain instructionsthat enable a context processor to respond to a specific query accordingto the expected information needs of the user. Thus, the context file902 d for the professional user may include query revision rules tomodify a received query such as “Nikon camera” to “Nikon DX2”, which isa current model of a professional digital SLR, and one deemed by thecontent provider to be of most interest to the professional user. Bycontrast, the context file 902 e for the consumer user may include queryrevision rules to modify this same query to “Nikon Coolpix 7600”, againa current model of the Nikon cameras, and determined by the contentprovider to be the best Nikon camera for a typical consumer user.Continuing this example then, the vertical content site would pass theconsumer context file 902 e to a context processor along with the userquery of “Nikon camera”, and the context processor would use the querymodification rules to generate the appropriate revised query forexecution.

The arrangement and interrelationship of the context files is highlyflexible and is decided by the particular vertical content provider.Each of the context files 902 can point to any number of other contextfiles 902 in an arbitrary graph manner, as best determined by thecontent provider. For example, the consumer user context file 902 ereferences two other context files, the “Looking for a Camera” contextfiles 902 h, and the “Shopping for a Camera” context file 902 i. Thesecontext files more precisely focus on serving the user's intention, theformer focusing on the information needs when a user is still lookingfor a camera and in need of information to evaluate potential products.The latter context is appropriate when a particular camera has beenselected and the user is now shopping for the camera based on price,availability, and other factors. Again, each of these context files 902references different and more selective contexts. Thus, the “Looking fora Camera” context file 902 h references a group of context files 902 kpertaining to various types of reviews of digital cameras. The “Shoppingfor a Camera” context file 902 i references context files 902 m, 9021for comparing prices, and for comparing vendors. The context files 902can also be arranged hierarchically through a series of directories.

As previously discussed, a context file may include query revisionrules, and search engine control information that enables the contextprocessor to programmatically tailor the user's query to the informationneeded, as indicated by the context. For example, once the user entersthe “Looking for a Camera” context, that context file 902 h may containsearch control data that selects specific websites that contain consumeroriented camera reviews, as deemed appropriate by the vertical contentprovider. This control data would thus be used by the search enginesystem to select one or more document collections for targeting thequery (or revised queries) thereto.

Similarly, the “Shopping for a Camera” context file 902 i would includesearch control data that selects various price comparison engines toobtain current market prices on a given camera. These examplesillustrate how selection of a context can programmatically vary thesearch query and search control data and parameters in order to bettersuit the user's information needs.

It is important to further point out here that the specific editorialdecisions reflected in each context file 902—how to revise a query basedon whether the user is a professional or a consumer, or which sites tosearch depending on whether the context is shopping or looking—are madeby each vertical content provider individually. This gives each verticalcontent provider—such as those with expertise in a particular field,such as digital cameras—the ability to define the contexts as they seefit, thereby using their own judgment, expertise, knowledge, andopinions to make the various determinations. Each vertical contentprovider can define very detailed and precisely crafted contexts, eachof which can specifically control the operations of the programmablesearch engine in responding to a search query. Users ultimately benefitfrom this individuated capability because the vertical content providersto create a dynamic information “market”: a market not merely forcontent itself, but for perspective, experience, and knowledge. That is,vertical content providers now offer users the ability to “search theworld” through their own point of view, as suggested in FIG. 1 by thetext “Search the web with digitalslr.org.”

Site/Page Annotation File

One mechanism for encapsulating the expertise and judgment of eachvertical content provider is, at least in part, the site/page annotationfile 900. This context file 900 includes information variouslycategorizing or describing characteristics of sites or pages on theInternet. Each entry in the site/page annotation file 900 provides anidentifier of a site or page, e.g., a URL, along with a number of tagsor token identifying attributes, characteristics, weightings, or otherqualitative or quantitative values. The tags can be explicitly typed(e.g., as <tag, value> pairs), or implicitly typed based on order anddata format. A URL can specify a site or page completely, or in part asa URL prefix, for some portion of a web site. Such an annotation file900 can be provided using existing standard formats such as RSS (RDFSite Summary or Really Simple Syndication).

The following are some examples of the contents of a site/pageannotation file:

url, http://www.dealtime.com/xPR-Nikon_D100~RD- 81887137412, descriptor,Review/NegativeReview, rank, 6, comment, “Professional Photographerlists various shortcoming and compatibility problems” url,http://www.dealtime.com/xPR-Nikon_D100~RD- 81887137412, descriptor,Review/ProfessionalPhotographerReview, rank, 0, comment, “ProfessionalPhotographer is less thrilled than many others about the D100” url,http://www.dpreview.com/reviews/read_opinion_text.asp?prodkey=nikon_d100&opinion=15851, descriptor, Action, rank, 0, comment,“Short review on using the D100 for sports photography” url,http://nikonimaging.com/global/news/, descriptor, News, rank, 3,comment, “Nikon's web site. Lots of info, but hard to navigate” url,http://www.kenrockwell.com/tech/2dig.htm, descriptor, Guide, rank, 0,comment, “Explains Digital SLRs vs Point and Shoots” url,http://www.luminous- landscape.com/tutorials/nikon-sn.shtml, descriptor,Review/ProfessionalPhotographerReview, rank, 8, comment, “Extremelydetailed, very technical, comparative review” url,http://www.photographyreview.com/, descriptor, Review, rank, 6, comment,“Good all around site for photography buffs” url,gallery.photographyreview.com/showphoto, descriptor, Photos, rank, 8,comment, “Good showcase of great photography with a wide range ofcameras” url, http://www.olympusamerica.com/, descriptor, Manufacturer,rank, 10, comment, “Olympus's web site. Well organized and informative”

In this embodiment of a site/page annotation file 900, each entry is aset of <name, value> pairs, as follows:

URL: provides the network address for where the site or page is located.Note that both specific pages within sites can be identified, as well ashome pages for large sites.

Descriptor: a semantic label describing the site or page. The contentprovider is free to use any labels he or she chooses, since the queryprocessing and post processing operations are written in terms of rulesthat can operate on these same descriptors. In the above example, thevertical content provider has labeled various sites/pages to theircontent type (e.g., “Negative review” or “News” or “Photos”), as well asto the type of entity which provides the information (e.g.,“Manufacturer”). Again, these descriptors are merely illustrative, andthe selection of which particular descriptors are used to describe asite will be dependent in at least in part on the particular category ortopic for the subject matter of the domain.

Referring back then first entry here is for a specifically identifiedpage on a remote site (dealtime.com) that contains a “negative review”of the Nikon D100 camera

The preprocessing and post processing operations can use the tags asconditions for evaluation. For example, a post processing rule in the“Negative Reviews” context file 902 n would select for inclusion in thesearch results that had a tag “Negative Review/NegativeReview”. Thevarious tags shown above—Manufacturer, Guide, Photos, etc.—are merelyillustrative of the scope and variety that can be used. The ability totag any site or page with a semantic label allows for very powerfulpre-processing and post processing operations by the context processor.

In one embodiment, there is provided a common ontology of tags which canbe used, either exclusively or in conjunction with a set of private tagsdefined by vertical content provider. The ontology includes a hierarchyof categories of information and content on Internet. One usefulontology is provided by the Open Directory Project, found at dmoz.org.All or a portion of such an ontology can be used for the tags. Theontology can be public, as in the OPD, or proprietary, or a combinationof both.

Rank: Each entry can have a rank (or “score”, “weight”, etc.) a figureof merit as to the importance, quality, accuracy, usefulness, and thelike of the particular page or site. This value is provided by thevertical content provider, again based on his or her own judgment andperspective. The rank value further allows the context processor toselectively include (or exclude) search results that have certain rankvalues, or to rank individual search results by this value as well.

Comment: Each entry can have a comment, explanation or description thatthe vertical content provider can use to further describe the page tothe user. The comment allows the vertical content provider to furtherarticulate the relationship between the page and the user's informationneed.

Note further, that a given site or page can have multiple entries in thesite/page annotation file 900, each with its own descriptors, and othertags. For example, the first two entries above are for the same page,but with different descriptors, ranks, comments and so forth. When morethan one entry matches a given URL, depending on the use, either both orthe most specific entry is applied.

The URL, Descriptor, Rank, and Comment fields are illustrative of thetypes of information that can be included in the site/page annotationfile 900. The vertical content provider can define any number of otheror additional attributes, and then define complementary pre-processingand post-processing rules that operate on such attributes. For example,other attributes that can be included in the site/page annotation fileinclude:

Content Type: a designation of the type of site or page, such as guide,scientific article, government report, white paper, thesis, blog, and soforth.

Source Type: a designation of the source of the document, which maybethe same or different than the Tag. For example: government, commercial,non-profit, educational, personal, and so forth. An “Organization”attribute may serve a similar purpose.

Location: a designation of the country, state, country or othergeographic region relevant to the page, using names, standardabbreviations, postal codes, geo-codes, or the like.

User Type: a designation of the intended type of user or audience forthe site or page. For example, lay person, expert, homemaker, student,singles, married, elderly, and so forth.

The foregoing descriptors are themselves instances or specializations ofa generic attribute type ‘tag’. Accordingly, vertical content providerscan choose to simply use the “tag” designation in association with aproperty value (e.g., tag, “Manufacturer”), or may use somespecialization of tag, such as those listed above, or a combination ofboth approaches. This feature further enhances the flexibility and theextensibility of the present invention.

Any given page or site can have multiple different entries in thesite/page annotation file. For example, the first two entries in theabove list are for the same page, but have different tags, the firstbeing a Negative Review, and the second being a ProfessionalPhotographer Review, different ranks, and different comments. Thisallows the vertical content provider to express the relevance of a givesite for a particular context, rather than being limited to a singleinclusion.

Knowledge Base

A second mechanism for capturing the knowledge and expertise of thevertical content provider is the knowledge base file 904. The knowledgebase file 904 is used to describe specific knowledge of concepts, facts,events, persons, and like. This information is encoded in a graph ofobject classes and instances thereof. A simple knowledge base file 904could be as follows:

<KB> <Class id=″CameraModel″/> <Class id=″DigitalSLRCamera″> <subClassOfref=″CameraModel″/> </Class> <DigitalSLRCamera id=″NikonD100″><manufacturedIn ref=”Japan”/> <name>D100</name> <name>Nikon D100</name><manufacturer>Nikon</manufacturer> <brand>Nikon</brand><format>SLR</format> <madein>Japan</madein> <modelyear>2003</modelyear><megaPixels>6mp</megaPixels> </DigitalSLRCamera> <DigitalSLRCameraid=″CanonDigitalRebel″> <manufacturedIn ref=”Japan”/><name>EOS300D</name> <name>Digital Rebel</name><manufacturer>Canon</manufacturer> <brand>Canon</brand><format>SLR</format> <madein>Japan</madein> <modelyear>2003</modelyear><megaPixels>6.5mp</megaPixels> </DigitalSLRCamera> </KB>

This knowledge base defines the class of “CameraModel”, used to identifyindividual types of cameras. Each class had a class id, as shown. Aclass can then be a subclass of another class. Hence, the class“DigitalSLRCamera” is a subclass of the “CameraModel” class.

Instances of a class can then be defined as well. Here, two differentinstances of the class “DigitalSLRCamera” are defined by giving it aspecific id, here “NikonD100” and “CanonDigitalRebel”, and a listing ofa variety of properties, such as their name, manufacturer, location ofmanufacture, model year, and so forth. The properties for each class aredetermined by the provider of the knowledge base file 904, such as thevertical content provider.

The programmable search engine may maintain its own global knowledgebase file as part of its global context files. This global knowledgebase can provide an extensive database encapsulating a vast array ofknowledge, concepts, facts, and so forth, as extracted from content onthe Internet, provided by experts or editors, or any taken from existingdatabases. Vertical content providers can then make use of this globalknowledge base by providing preprocessing and post processing operationsthat make use of such knowledge base information, as further describedbelow.

Pre-Processing and Post Processing Context Processing Operations

The context files 902 use a script or markup language to define thevarious pre-processing, search engine control, and post-processingoperations. The various elements of the language are as follows:

(i) Object Evaluation

The knowledge base file 904 can be used to evaluate whether particularobjects have defined properties or attributes. In general, there arethree basic types of objects that can be evaluated related to theknowledge base: queries, users, and search results. The form of theevaluation commands are generally the same.

The query evaluation commands for evaluating terms using the knowledgebase file 904 are as follows:

<query.denot.property>property_value</query.denot.property><query.denot.InstanceOf>class_id</query.denot.InstanceOf><query>query_term</query>

The first type of term based evaluation is used to evaluate whether theconcept expressed by one or more query terms matches some object in theknowledge base file that has the specified property with the specifiedproperty_value. The context processor processes this command bytraversing the knowledge base file 904 (as a graph, for example) untilit finds an object having a property with the matching property value.For example, assume the knowledge base file 904 portion described above,and the query evaluation command:

<query.denot.Manufacturer>Nikon</query.denot.Manufacturer>

and the input search query “D100”.

Here, the query term “D100” matches the name of a camera instance in theknowledge base file 904. The context processor than checks whether theManufacturer property of that instance is “Nikon”. Since it is, thequery “D100” is said to denote a camera manufactured by Nikon, even ifthat is not specifically disclosed in the query term itself. Accordinglythe query evaluation command is satisfied, and the context processorwould then take an appropriate action that was dependent on thisevaluation. As will be further illustrated below, a variety of differentcommands to the context processor can be made conditional based on theevaluation of the query evaluation command.

The second type of query evaluation command is query.denot.InstanceOf.This command is evaluated to determine whether a particular queryindicates that an instance of a class has been described in the query,rather than property. For example, consider the query evaluationcommand:

<query.denot.InstanceOf>DigitalSLRCamera</query.denot>

where the user query is “8 mp SLR”.

Here, the query is decomposed into terms “8 mp” and “SLR”, and these arechecked against the property values for the objects in the knowledgebase file. In this example, these properties match the properties forthe Nikon D100 camera, satisfying the query evaluation command. Again,the context processor would undertake whatever command was conditionedon the evaluation command.

The last type of query evaluation command <query>query_term</query> isthe simplest. The query evaluation command is satisfied if an inputsearch query term matches the query_term.

As noted above, the context files may use combination of queryevaluation commands as conditional triggers for further contextprocessing. Example of these will be further described below.

As with the evaluation of queries, so too can users and search resultsbe evaluated for their properties, with respect to any defined class inthe knowledge base file. Thus, the attributes of user can be evaluatedwith the following command

<user.property>property_value</user.property>

where property refers to any available property of the user, such asuser name, login, account number, location, IP address, site activityand history (e.g., clicks, focus, page dwell time) and so forth. Some ofthese properties can be locally available from the knowledge base file904. Alternatively, the property information can be extracted (e.g.,queried) from any accessible legacy database (e.g., a customer database,account database, registration database, or other data source), whichexports an appropriate programmatic interface. Other properties, such assite activity, are made available from site tracking tools that monitoreach user's activity at the vertical content site.

Users can also be evaluated for membership in classes, using thefollowing:

<user.InstanceOf>class_id</user.instanceOf>

Here, a class of users (e.g., “Professional”) can be defined in theknowledge base file 904, and the properties of the current user comparedby the context processor against the properties of an identified classfor match in values. If a property match is found, the user is deemed amember of the class.

Similarly, any search result can be evaluated as well, as to itsproperties, as defined in either the source/page annotation file 900 (oralternatively, in its metatags). Here, the evaluation command would takethe form:

<result.tag>tag_value</result.tag><result.tag.InstanceOf>class_id</result.tag.InstanceOf>As a default <result.tag> may be abbreviated to <tag>.

In the first command, a given search result (or set thereof) can beevaluated with respect to its properties, such as content type, date,source, user type, etc. This outcome of the evaluation can be used tocontrol further context processing. Similarly, search results can beevaluated using the second command syntax to determine if they areinstances of various classes defined in the knowledge base file 904.

These following context processing operations can be executedunconditionally, or conditionally based on any of the foregoing types ofevaluation operations (e.g., evaluations of query terms, users, orsearch results).

(ii) Query Modification

There are two basic types of query modification rules, those thataugment or add terms to a query, and those that replace query terms. Thefollowing is example syntax for the query modifier command:

<QueryModifier type=″augment″ value=″query term″/> <QueryModifiertype=″replace″ query=”query term” value=″replacement term″/>

The type attribute defines either an augmentation or replacement typequery modification. The value attribute includes the query term that isto be added to the user's original input search query, or that is toreplace the input search query. The query attribute is optional. Ifpresent, then the context processor scans the search query and replacesthe any term matching the query term with the replacement term. This isuseful, for example, to correct misspellings, expand abbreviations (orcontrawise use abbreviations in place of terms), and other in placeadjustments. If the query attribute is missing, then the entry querystring is replaced by the replacement term. Of course, the replacementterm can include any number of terms.

Query modification can made conditional on any of the evaluationcommands. For example:

<QueryModifier type=“augment” value=“Digital SLR”><query.denot.InstanceOf>DigitalSLRCamera</query.denot > </QueryModifer>

This example would reformulate a query, say the query “D100” to includeanother query “Digital SLR” since the term “D100” denotes an instance ofa digital SLR camera, according to the knowledge base file 904.

As another example:

<QueryModifier type=“augment” value=“Professional reviews”>   <user.property>professional</user.property> </QueryModifer>

In this example, assume again the user's query is “D100.” Here, theproperties of the current user are evaluated. If the user is determinedto be “professional”, based on properties available from the browser,site activity history, login and password, etc. For example, if the useraccess a number of pages in the vertical content site dedicated toprofessional or expert level information (e.g., detailed technicalpages), then the user may be inferred to be a “professional” user, eventhough no other information is known about the user's identity. In thiscase, the query is reformulated to include the term “professionalreviews” even though the user did not include these terms in the query.

These are but a few examples of a how a vertical content provider canextend and improve the user's queries based on his own expertise and theflexible context processing operations.

(iii) References to Related Contexts

A context file 902 can reference or include another context file 902, asdescribed above, to form an arbitrary graph of connections. Severalelements are used for referencing context files.

A context file can include another context file, as follows:

<include scr=“path name”>

The include command references another context file 902 as beingincluded in the current context file. The context processor will readthe included context file and process all of the instructions therein.Pathname identifies the location of included context file 902. Includedcontext files 902 can be used for any type of context processingoperation.

A context file can also identify a related context file, as follows:

<relContext href=“path name”>  <anchorText>contextdescription<anchorText> </relContext> and <relContext href=“pathname”>context description</relContext>

The relContext command identifies a related context for the currentcontext file. The relContext command can be used in both pre-processingand post-processing operations. Examples of the use of related contextsin post-processing operations are illustrated in FIG. 9, and in FIGS. 2and 3. The context description is anchor text that the user will see inthe browser. When selected, the identified related context file isretrieved and processed. The first type of related context command isused to define related contexts for varying types of information needs.FIG. 2 illustrates this type of related context via related contextlinks 204. The first link 204 there is associated with a related contextfile 902 (e.g., context file 902 h) that includes the followinginstructions:

<relContext href=“ /chooseCamera”>    <anchorText>If you are trying todecide which camera to buy . . .</anchorText> </relContext>

This command is processed by the context processor when the link 204 onthe anchor text is selected, and the corresponding context file“cameras/chooseCamera” is retrieved and processed. The resulting page isillustrated in FIG. 3.

The relContext command may also be used with the various types ofevaluation commands, to make the reference to the related contextconditional. For example:

<relContext href=“ /chooseCamera”>   <query.denot.instanceOf>DigitalSLRCamera</query.denot    .instanceOf>   <anchorText>If you are trying to decide which camera to buy...</anchorText> </relContext>

Here, the related context DigitalSLRCamera is accessed here only if thequery.denote command evaluates true, that is where the query termsdenote an instance of a model of digital camera listed in the knowledgebase file 904. Similar conditional evaluations can be based on theproperties of the user or the properties of the search results.

The second type of related context command is used to define relatedcontexts that appear as annotations in conjunction with search results.This type of related context is illustrated in FIG. 2 by related contextlinks 206. For example, the related context file 902 h that generatedFIG. 2 also includes the following instructions:

<relContext href=“cameras/Manufacturer”>More ManufacturerPages</relContext>

Here, the anchor text “More Manufacturer Pages” is then linked to theassociated context file 902, which contains further instructions tosearching and displaying pages for digital camera manufacturers.

The relContext command takes as an href any valid URL, and thus, canalso reference any available Internet site. For example, the relContextcommand can directly link to an online encyclopedia or dictionary toprovide an annotation for a search result that would provide a detailedexplanation of the result.

In pre-processing operations, a second type of cross reference torelated context is used, context redirection. The command format for thecontext redirection command is as follows:

<contextRedirect href=“pathname”>redirectioncondition*</contextRedirect>

Again, pathname indicates the location of another context file to beprocessed if certain redirection conditions are met. The redirectionconditions (one or more as indicated by “*”) can be based on anyavailable information about the query (e.g., query terms, or informationdependent thereon), the user (e.g., IP address, login, site clickthrough history, prior purchases), or other programmatically availableinformation.

In one embodiment the redirection conditions can be based on the anyevaluation commands previously discussed:

<query.denot.property>property_value</query.denot.property><query.denot.InstanceOf>class_id</query.denot.InstanceOf><query>query_term</query> <user.property>property_value</user.property><user.InstanceOf>class_id</user.instanceOf><result.tag>tag_value</result.tag><result.tag.InstanceOf>class_id</result.tag.InstanceOf>

For example, assume the knowledge base file 904 portion described above.Further, assume the redirection command:

<contextRedirect href=“Nikon_cameras”>   <query.denot.Manufacturer>Nikon</query.denot.Manufacturer></contextRedirect>and the input search query “D100”.

As above, the query evaluation command is positively evaluated, sincethe query term “D100” matches the name of a camera instance in theknowledge base file 904, which instance has the Manufacturer propertyvalue “Nikon”. The context processor thus executes the contextredirection command and accesses the context file “Nikon_cameras” forfurther processing. This capability allows the vertical content providerto his or her own knowledge base to analyze queries and reformulate themon behalf of the user.

The user evaluation user.InstanceOf can likewise be used to redirectcontext processing based on the particular user properties. For example,consider the redirection command:

<contextRedirect href=“NegativeProfessionalReviews”>   <user.InstanceOf>Professional_User</user.InstanceOf></contextRedirect>

Here, the properties of the user can be ascertained from the knowledgebase file 904, and other information as described (e.g., site history).If the user is determined to be a professional user, then the contextprocessor accesses and processes the NegativeProfessionalReviews contextfile.

As mentioned, any number of redirection conditions (e.g. evaluations)can be used together in a context redirection command such as:

<contextRedirect href=“Recommended_SLR_cameras”>   <query.denot.megapixels>6mp</query.denot.megapixels>   <query.denot.megapixelsmatchType=“greaterThanOrEqualTo”>6mp</query.denot.megapixels>   <query.denot.megapixelsmatchType=“lessThanOrEqualTo”>8mp</query.denot.megapixels>   <query.denot.modelyear>2005</query.denot.modelyear></contextRedirect>which would effect the context redirection only when all of theredirection conditions are satisfied, e.g., for a query containing theterms which denote digital SLR with between 6 mp and 8 mp, for the 2005model year.

The context redirection is particularly powerful when combined with thequery modification rules, previously discussed. A vertical contentprovider can define a number of context redirections based on queryterms, each of redirects the context processor to an appropriate contextfile, depending on say, whether the query denotes shopping for a cameraversus seeking customer warranty information. In the respective targetcontext files, specific query modification rules would then be processedto reformulate the query as most appropriate given the identifiedcontext.

(iv) Restriction

In post processing operations, the context files can be used to controlthe scope, number, or types of results and entries that are provided tothe user. To this end, the context files can include conditionalinstructions that define various types of restrictions (e.g., filters).These restrictions are provided by the restriction command. This commandhas the following syntax:

<Restriction count=“n”>     restriction condition*     restrictionaction* </Restriction>

The restriction condition operates in a similar manner to theredirection condition previously discussed. Here, the restrictioncondition is evaluated with respect to the attributes (tags), if any,associated with the search results, as compared to the entries in thesite/page annotation file. Any attribute (or set of attributes) can beused as restriction conditions, such as the type, source, year,location, of a document or page, to name but a few. The contextprocessor receives the search results (here a set of candidate searchresults) from the search engine, and compares each candidate result (beit a site, page, media page, document, etc.) with the entries listed inthe site/page annotation file 900. Only those candidate results whichare listed in the annotation file 904 and have the specified matchingattributes are included in the context processed search results. Therestriction count is an optional parameter and indicates how many of thematching results are to be included in the context processed searchresults. If left out, then all matching results are included.

The restriction action is an optional parameter that specifies a furtheraction to take if the restriction condition is met. This actionincludes, for example, annotating the search results with a link to arelated context (using the relContext command), such as links 206illustrated in FIG. 2.

Consider the following example:

<Restriction count=“2”>   <descriptor>Review</descriptor>  <rank>5+</rank>   <relContext href=“Reviews”>More Review</relContext></Restriction> <Restriction count=“2”>   <descriptor>Guide</descriptor>  <rank>5+</rank>   <relContext href=“Guides”>More Guides</relContext></Restriction>

Assume that the search query was a general query on “digital cameras”,and that the search results returned 1,000,000 pages covering everythingfrom manufacturers and retailers of digital cameras, to online userforums and services for printing photographs. Since the user's searchwas quite general, the vertical content provider can use the postprocessing to provide a selection of a number of different types ofsearch results, as illustrated, for example in FIG. 2. In processing theabove code example then, the first restriction command causes thecontext processor to select the first two search results that havematching entries (i.e., matching URLs or portions thereof) in thesite/page annotation file 900 and include the descriptor “Review”. Thecontext processor also uses the restriction action for the relatedcontext, to annotate these two search results with a link to relatedcontext file “Reviews”, with the link labeled “More reviews.” FIG. 2shows an example of such annotation link 206.

The second restriction causes the context processor to select the firsttwo search results that have matching entries in the site/pageannotation file and include the descriptor “Guide.” The contextprocessor would then use the restriction action to annotate theseresults with a link to the related context file “Guides.”

As mentioned previously, the context processing operations canundertaken by multiple different entities in the system, including atthe client device, the vertical content site, and the programmablesearch engine, each using their own locally available context files.Thus, all of the above describe features can be effectively integratedwithin and between different system entities. For example, a verticalcontext provider can define a context file that defines various contextredirections using the redirection condition based on the globalknowledge base files. This enables the vertical content provider toleverage the global knowledge base, but add their own personalperspective and judgment to its underlying facts.

In post processing operations, the context file 900 can be used tocontrol just the annotations that appear on a search result, withoutchanging the actual order of the search results. To this end, thecontext files 900 can include conditional instructions that definevarious types of annotations. These annotations are provided by theannotate command. This command has the following syntax:

<Annotate count=“n”>   annotation condition*   annotation action*</Annotation>

The annotation condition operates in a similar manner to the restrictioncondition previously discussed. Here, the annotation condition isevaluated with respect to the attributes (tags), if any, associated withthe search results, as compared to the entries in the site/pageannotation file. Any attribute (or set of attributes) can be used asannotation conditions, such as the type, source, year, location, of adocument or page, to name but a few. The context processor receives thesearch results from the search engine, and compares each result (be it asite, page, media page, document, etc.) with the entries listed in thesite/page annotation file 900. Results that satisfy the condition areannotated with the annotation action. Unlike the Restriction command,the Annotate command does not cause any search result to appear or notappear in the search results. Annotate commands can be used bythemselves or in combination with any of the other commands, includingRestrictions.

In a very simple implementation, the context file is left implicit andonly consists of Annotation commands, where each result that is assigneda tag/label/armotation by the annotation files is annotated with thatlabel/annotation. Further, the user may be ‘subscribed’ to a number ofannotation files or ‘feeds’, all of which are applied to the user'ssearch results. In a further twist, the user can also indicate that hewould like the feeds used by another user to also be applied to him.

(v) Search Engine Control Data

Finally, context files 902 can contain instructions that control theoperation of the programmable search engine itself in terms theselection of which particular document collections to be searched, andvarious algorithmic or parametric settings for the search engine.Selection of a document collection for searching is provided by thefollowing command:

<Corpus ref=“document_collection”>     //other context operations//</Corpus>

The corpus command takes as its argument a reference to the name (orULR) or a selected document collection. The document collection name ismapped (either locally, or by the programmable search engine) todocument collection and corresponding index available to theprogrammable search engine (e.g. particular index in the contentserver/index 870).

The corpus command can be made conditional using any of the foregoingdescribed evaluation commands, as well as including any of therestriction, redirection, related context, and so forth. For example, aparticular document collection may be selected where the query isdetermined using the evaluation commands to include certain keywords orinstances of objects in the knowledge base. Thus, a query that isevaluated to include a query term denoting a scientific term, like“Heloderma suspectum”, or a medical term, would then cause a selectionof an appropriate scientific literature database.

Control of search engine parameters is via the SearchControlParamsoperations. In general, most modern search engines use a number ofdifferent attributes of a search query and the individual indexeddocuments (e.g., frequencies of terms in URL, anchor text, body, pagerank etc.) to determine which documents best satisfy the query. Thedocuments are then ranked accordingly. A ranking function is essentiallya weighted combination of the various attributes. Normally, the weightsof the attributes are fixed, or at least not externally controllable bythird parties. The SearchControlParam however gives vertical contentproviders access to these weights. The syntax is as follows:

<SearchControlParams>   <attribute-name>weight</attribute-name>  <attribute-name>weight</attribute-name>   ... </SearchControlParams>

Here, attribute-name is the name of the particular attribute used by thesearch engine to calculate a relevance ranking. The specific attributenames are disclosed by the programmable search engine provider, sincethey are internal to that provider's own engine. Typical attributes, asindicated above including term frequency in URL, term frequency in body,term frequency in anchor text, term frequency in markup, page rank. TheSearchControlParams operator can work with any exposed attribute orparametric control of a programmable search engine, and thus theforegoing are understood to be merely exemplary. The weights used inthis operator can be either normalized or non-normalized, and in thelatter case, the input weights can be internally normalized by thecontext processor or by the search engine itself. A vertical contentprovider need not specify weights for all the attributes the searchengine uses, but only those of interest to the provider of the contextfile.

The context files may take various embodiments. In the some embodiments,the context files are individual files stored in a file system. In otherembodiments, the context files are stored in a database system, again aseither separate files, or of database entries, tables or otherstructures. For example, a context file in database embodiment may bestored as a collection of context records for a identified source (e.g.,a specific vertical content provider), a type (e.g., knowledge base,site/page annotation, etc.), associated commands (e.g., evaluation,restriction, redirection, relation, annotation, etc.), and remainingattributes and conditions. Accordingly, no limitation is imposed on theunderlying implementation of the context files by the present invention.

Detecting Biased Contexts

As described above in connection with FIG. 8, the PSE system 800provides context processing for queries directly received from clientdevices, that is, queries that are not forwarded via a search engineinterface from a vertical content provider. In handling such queries,the PSE system 800 makes use of the global context files 842, and canalso use the cached context files 840 as a source of global contextinformation, for example to annotate results, provide links to relatedcontexts or other sites, and to restrict search results to particulargroups. Because the global context files 842 can derive contextprocessing operations from the cached context files, it is expected thatsome third parties will attempt to manipulate the PSE system 800 toprovide biased or “spam” type results to user's queries.

For example, a user searching for “digital cameras reviews” may receivea set of search results that include links (e.g., such as relatedcontext links 306) to related contexts provided by third parties such asreviews and other technical information. However, the provider of thecontext files may program its links 306 to include links to certainproduct vendors (e.g., those who have paid for such inclusion), usingany variety of the query modification commands (e.g., replacing the“digital camera reviews” query with the name of the vendor), relatedcontext commands, or redirection commands. As a result, when the useraccesses the related context links 306, the user receives “spamresults”. While such results may be desirable when the user's query isprovided from a vertical content provider site, it is desirable todetect such biased contexts when the user is accessing the PSE system800 directly and thereby prevent unscrupulous spamming of user queries.

A number of different mechanisms can be used individually or together todetect biased and spam related context files and vertical contentproviders. These include the following.

First, an offline analysis of the context files obtained from eachvertical content provider is performed to identify sites or pages knownto be spam-related. FIG. 10 illustrates the basic processing model foroffline analysis. Some content providers will use their own site/pageannotation file 800 when defining pre-processing or post processingoperations, for example for annotating links 306 next to search resultsor defining restrictions operations. Other content providers will use athird parties site/page annotation file 800 (that is, one located in adifferent host system).

Accordingly, each a site/page annotation file received from a verticalcontent provider 804 (whether by crawling from context file crawler 880or by registration interface 860) is processed with a spam filter 892.The spam filter 892 analyzes each of the pages listed in the annotationfile to determine whether the page is a spam page. The spam filter 892may be dynamic (e.g., algorithmic determination of spam from contentanalysis) or static (e.g., comparison of the page with a list of knownspam pages). Spam filter that may be used here include those describedin U.S. patent application Ser. Nos. 10/921,381 and 11/004,250,incorporated by reference above. If a significant portion (e.g., morethan 40%) of the pages listed in a site/page annotation file of aparticular vertical content provider is determined by the spam filter tobe spam, then one of two actions can be taken. First, if the site/pageannotation file is provided by the vertical content provider itself,then this vertical content provider is deemed a biased vertical contentprovider, and its context files are not used for further direct contextprocessing as part of the global context files 842. As a result, searchresults from direct queries to the PSE system will not include links,annotation, or other content from this particular vertical contentprovider. Alternatively, if the site/page annotation file is source fromanother domain, then instead the specific documents deemed to be spamare stripped during post-processing from search results or linksprovided by this context file.

This first stage of offline processing thus operates to detect andeliminate vertical content providers and/or context files that are knownto be spam related.

A second stage is spam and bias detection during direct queryprocessing. FIG. 11 illustrates the basic processing model for querytime analysis. In some embodiments, it is beneficial to distinguishbetween spam related results (and contexts) and biased results (andcontexts). Spam results may be characterized (without limitation) ascommercially influenced communications that are not reflective of theuser's actual interests; biased results may be characterized (again,without limitation) as being sufficiently different in content fromsearch results produced by objective search models.

Accordingly, in one aspect of query processing, spam-related contextfiles are identified, particularly those that include query modificationcommands that result in a spam query, and hence heavily spam orientedsearch results. Here, the query 1100 is received from the client device802, and context processing is applied. The search engine 850 executesthe search on the context processed query 1102, as described above. Someportion of the search results 1106 resulting from context processingusing a particular vertical content provider's context files areprocessed with a spam filter 892 to determine an average spam score forthe context processed search results 1106. For example, the top 10% ortop 1,000 search results can be processed with a spam filter 892. In oneembodiment, an average spam score is computed by the spam filter 892from the spam score of such search result. If the average spam score isover a predetermined threshold (dependent on the particular spamfilter), then the vertical content provider is identified as a spamcontent provider. In another embodiment, the percentage of searchresults determined to be spam is compared with a predetermined threshold(e.g., 40%) and if this percentage exceeds the threshold then thevertical content provider is again identified as a spam contentprovider. In either embodiment, the spam search results, as well links,annotations, or other context related content from the vertical contentprovider's context files are not included in the filtered results 1112.

In the second aspect of query processing, biased search results areidentified. Here, bias results are those that are not necessarily spamresults, but rather are sufficiently different from non-contextprocessed search results that the vertical content provider is deemed tobe biased in its manipulation of the context files.

In this second aspect, the search results resulting from the contextprocessed query 1106 are compared by a bias detection algorithm 894 withthe search results 1108 from the original (or native) search query 1104to determine a distance measure between the two sets of the searchresults. The native search results are assumed to be unbiased and beingmost relevant to the original search query. If the two sets of searchresults are identical, then the distance between the result sets iszero, and the provider of the context files is deemed to be unbiased.However, if the distance measure between the two sets of search resultsis significant (e.g., over threshold amount), the vertical contentprovider is deemed biased. In this case, the provider's context filesare not used in post-processing, and thus the filtered results 1112 willnot include any links, annotations, or other context related contentsfrom the provider.

The distance measure for measuring the distance between the native andcontext processed search results can be any number of set comparisonmeasures. One distance measure is the percentage of context processedsearch results that are the same as native search results. Variations inthe required degree of sameness can be used, resulting in differentdistance measures. Thus, the sameness criteria can be flexibly definedto range from being the same identical documents/pages, to being fromthe same host site, to being different versions of the samedocument/page on the same or different sites.

Other variations of the spam detection at query processing may also beused. In one embodiment, instead of comparing the average spam scorewith a predetermined threshold, the average spam score is insteadcompared with an average spam score of the search results from a native(non-context processed) search. If the average spam score for thecontext processed search results is significantly higher than the nativespam score, then the context file is deemed biased, and is not used forpost processing the search results.

The present invention has been described in particular detail withrespect to one possible embodiment. Those of skill in the art willappreciate that the invention may be practiced in other embodiments.First, the particular naming of the components, capitalization of terms,the attributes, data structures, or any other programming or structuralaspect is not mandatory or significant, and the mechanisms thatimplement the invention or its features may have different names,formats, or protocols. Further, the system may be implemented via acombination of hardware and software, as described, or entirely inhardware elements. Also, the particular division of functionalitybetween the various system components described herein is merelyexemplary, and not mandatory; functions performed by a single systemcomponent may instead be performed by multiple components, and functionsperformed by multiple components may instead be performed by a singlecomponent.

Some portions of above description present the features of the presentinvention in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. These operations, while describedfunctionally or logically, are understood to be implemented by computerprograms. Furthermore, it has also proven convenient at times, to referto these arrangements of operations as modules or by functional names,without loss of generality.

Unless specifically stated otherwise as apparent from the abovediscussion, it is appreciated that throughout the description,discussions utilizing terms such as “calculating” or “determining” or“identifying” or the like, refer to the action and processes of acomputer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (electronic)quantities within the computer system memories or registers or othersuch information storage, transmission or display devices.

Certain aspects of the present invention have been described usingcommands, mnemonics, tokens, formats, syntax, and other programmingconventions. The particular selection of the names, formats, syntax, andlike are merely illustrative, and not limiting. Those of skill in theart can readily construct alternative names, formats, syntax rules, andso forth for defining context files and programming the operations aprogrammable search engine via context processing.

Certain aspects of the present invention include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the present inventioncould be embodied in software, firmware or hardware, and when embodiedin software, could be downloaded to reside on and be operated fromdifferent platforms used by real time network operating systems.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored on acomputer readable medium that can be accessed by the computer. Such acomputer program may be stored in a computer readable storage medium,such as, but is not limited to, any type of disk including floppy disks,optical disks, CD-ROMs, magnetic-optical disks, read-only memories(ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic oroptical cards, or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus.

The algorithms and operations presented herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct morespecialized apparatus to perform the required method steps. The requiredstructure for a variety of these systems will be apparent to those ofskill in the art, along with equivalent variations. In addition, thepresent invention is not described with reference to any particularprogramming language. It is appreciated that a variety of programminglanguages may be used to implement the teachings of the presentinvention as described herein, and any references to specific languagesare provided for disclosure of enablement and best mode of the presentinvention.

Finally, it should be noted that the language used in the specificationhas been principally selected for readability and instructionalpurposes, and may not have been selected to delineate or circumscribethe inventive subject matter. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting, of the scopeof the invention, which is set forth in the following claims.

I claim:
 1. A computer-implemented method, comprising: receiving asearch query input and identification of a context file from athird-party content provider, wherein the third-party content provideris configured to receive the search query input from a client device andis configured to determine the identification of the context file basedon the search query input; processing the context file to identify oneor more commands for processing the search query input and one or morecommands for processing search results wherein one or more of thecommands for processing the search query input each specifies arespective precondition for evaluation of another of the commands forprocessing the search query input; processing the search query inputaccording to the one or more commands for processing the search queryinput to produce one or more context processed search queries; obtaininga plurality of search results responsive to the context processed searchqueries; processing the search results according to the one or morecommands for processing search results to produce a plurality of contextprocessed search results; analyzing, for each of one or more of thecontext processed search results of the plurality of context processedsearch results, whether a corresponding entry in an annotation filerefers to spam; removing from the plurality of context processed searchresults one or more of the context processed search results that eachhave a corresponding entry in the annotation file that refers to spam tocreate a plurality of modified search results; and providing themodified search results to the client device.
 2. The method of claim 1,wherein the context file refers to one or more related context files,and wherein processing the search query input according to the one ormore commands for processing the search query input further comprises:processing the search query input according to one or more additionalcommands for processing the search query input in the one or morerelated context files.
 3. The method of claim 1, wherein processing thesearch query input according to the one or more commands for processingthe search query input further comprises: evaluating a property value ofa query with respect to a predefined property; or modifying the searchquery input.
 4. The method of claim 1, wherein each of the entries inthe annotation file comprises a respective link to a respective webpage, the method further comprising determining whether the respectiveweb page contains spam.
 5. A system comprising: one or more computers;and a computer-readable medium coupled to the one or more computershaving instructions stored thereon which, when executed by the one ormore computers, cause the one or more computers to perform operationscomprising: receiving a search query input and identification of acontext file from a third-party content provider, wherein thethird-party content provider is configured to receive the search queryinput from a client device and is configured to determine theidentification of the context file based on the search query inputs;processing the context file to identify one or more commands forprocessing the search query input and one or more commands forprocessing search results wherein one or more of the commands forprocessing the search query input each specifies a respectiveprecondition for evaluation of another of the commands for processingthe search query input; processing the search query input according tothe one or more commands for processing the search query input toproduce one or more context processed search queries; obtaining aplurality of search results responsive to the context processed searchqueries; processing the search results according to the one or morecommands for processing search results to produce a plurality of contextprocessed search results; analyzing, for each of one or more of thecontext processed search results, whether a corresponding entry in anannotation file refers to spam; and removing from the plurality ofcontext processed search results one or more of the context processedsearch results that each have a corresponding entry in the annotationfile that refers to spam to create a plurality of modified searchresults; and providing the modified search results to the client device.6. The system of claim 5, wherein the context file refers to one or morerelated context files, and wherein processing the search query inputaccording to the one or more commands for processing the search queryinput further comprises; processing the search query input according toone or more additional commands for processing the search query input inthe one or more related context files.
 7. The system of claim 5, whereinprocessing the search query input according to the one or more commandsfor processing the search query input further comprises: evaluating aproperty value of a query with respect to a predefined property; ormodifying the search query input.
 8. The system of claim 5, wherein eachof the entries in the annotation file comprises a respective link to arespective web page, the method further comprising determining whetherthe respective web page contains spam.
 9. A computer storage mediumencoded with a computer program, the program comprising instructionsthat when executed by one or more computers cause the one or morecomputers to perform operations comprising: receiving a search queryinput and identification of a context file from a third-party contentprovider, wherein the third-party content provider is configured toreceive the search query input from a client device and is configured todetermine the identification of the context file based on the searchquery input; processing the context file to identify one or morecommands for processing the search query input and one or more commandsfor processing search results wherein one or more of the commands forprocessing the search query input each specifies a respectiveprecondition for evaluation of another of the commands for processingthe search query input; processing the search query input according tothe one or more commands for processing the search query input toproduce one or more context processed search queries; obtaining aplurality of search results responsive to the context processed searchqueries; processing the search results according to the one or morecommands for processing search results to produce a plurality of contextprocessed search results; analyzing, for each of one or more of thecontext processed search results of the plurality of context processedsearch results, whether a corresponding entry in an annotation filerefers to spam; and removing from the plurality of context processedsearch results one or more of the context processed search results thateach have a corresponding entry in the annotation file that refers tospam to create a plurality of modified search results; and providing themodified search results to the client device.
 10. The computer storagemedium of claim 9, wherein the context file refers to one or morerelated context files, and wherein processing the search query inputaccording to the one or more commands for processing the search queryinput further comprises: processing the search query input according toone or more additional commands for processing the search query input inthe one or more related context files.
 11. The computer storage mediumof claim 9, wherein processing the search query input according to theone or more commands for processing the search query input furthercomprises: evaluating a property value of a query with respect to apredefined property; or modifying the search query input.
 12. The methodof claim 1, wherein the annotations comprise labels, the method furthercomprising: receiving a selection of one of the labels; and generatingnew search results, wherein the new search results are associated withmetadata that includes the label.
 13. The method of claim 1, whereinprocessing the search query input according to the one or more commandsfor processing the search query input further comprises: selecting adocument collection for searching with the search query input;establishing a parameter of a search engine for processing the searchquery input; or filtering the search results to include only searchresults of a designated type in the context processing search results.14. The system of claim 8, wherein the annotations comprise labels andthe operations further comprise: receiving a selection of one of thelabels; and generating new search results, wherein the new searchresults are associated with metadata that includes the label.
 15. Thesystem of claim 5, wherein processing the search query input accordingto the one or more commands for processing the search query inputfurther comprises: selecting a document collection for searching withthe search query input; establishing a parameter of a search engine forprocessing the search query input; or filtering the search results toinclude only search results of a designated type in the contextprocessing search results.
 16. The computer storage medium of claim 9,wherein each of the entries in the annotation file comprises arespective link to a respective web page, the method further comprisingdetermining whether the respective web page contains spam.
 17. Thecomputer storage medium of claim 16, wherein the annotations compriselabels and the operations further comprise: receiving a selection of oneof the labels; and generating new search results, wherein the new searchresults are associated with metadata that includes the label.
 18. Thecomputer storage medium of claim 9, wherein processing the search queryinput according to the one or more commands for processing the searchquery input further comprises: selecting a document collection forsearching with the search query input; establishing a parameter of asearch engine for processing the search query input; or filtering thesearch results to include only search results of a designated type inthe context processing search results.
 19. The method of claim 1,wherein determining that one or more of the context processed searchresults that have corresponding entries in the annotation file refer tospam further comprises: identifying a related context file containing aplurality of annotations; and determining which of the annotations referto spam.
 20. The system of claim 5, wherein determining that one or moreof the context processed search results that have corresponding entriesin the annotation file refer to spam further comprises: identifying arelated context file containing a plurality of annotations; anddetermining which of the annotations refer to spam.
 21. The computerstorage medium of claim 9, wherein determining that one or more of thecontext processed search results that have corresponding entries in theannotation file refer to spam further comprises: identifying a relatedcontext file containing a plurality of annotations; and determiningwhich of the annotations refer to spam.